COMBINATORIAL HOPF ALGEBRAS, NONCOMMUTATIVE 
HALL-LITTLEWOOD FUNCTIONS, AND PERMUTATION 

TABLEAUX 

JEAN-CHRISTOPHE NOVELLI, JEAN- YVES THIBON, AND LAUREN K. WILLIAMS 

Abstract. We introduce a new family of noncommutative analogues of the Hall- 
Littlewood symmetric functions. Our construction relies upon Tevlin's bases and 
^\ \ simple g-deformations of the classical combinatorial Hopf algebras. We connect 

^^ ' our new Hall-Littlewood functions to permutation tableaux, and also give an exact 

^^ I formula for the q-enumeration of permutation tableaux of a fixed shape. This gives 

an explicit formula for: the steady state probability of each state in the partially 
Ch I asymmetric exclusion process (PASEP); the polynomial enumerating permutations 

^ ' with a fixed set of weak excedances according to crossings; the polynomial enumer- 

ating permutations with a fixed set of descent bottoms according to occurrences of 
the generalized pattern 2 — 31. 
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1. Introduction 



The combinatorics of Hall-Littlewood functions is one of the most interesting as- 
5^ I pects of the modern theory of symmetric functions [27j. These are bases of symmetric 

functions, depending on a parameter q which was originally regarded as the cardinal- 
ity of a finite field. They are named after Littlewood's explicit realization of the Hall 
algebra in terms of symmetric functions, which gave meaning to arbitrary complex 
values of q [24j . 

Combinatorics entered the scene with the observation by Foulkes that the transi- 
tion matrices between Schur functions and Hall-Littlewood P-functions seemed to be 
given by polynomials, which were nonnegative g-analogues of the well-known Kostka 
numbers, counting Young tableaux according to shape and weight. The conjecture 
of Foulkes was established by Lascoux and Schiitzenberger [21j , who introduced the 

1 
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charge statistic on Young tableaux to explain the powers of q. Almost simultane- 
ously, Lusztig [25] obtained an interpretation in terms of the intersection homology 
of nilpotent orbits, and it is now known that these Kostka-Foulkes polynomials are 
particular Kazhdan-Lusztig polynomials associated with the affine Weyl groups of 
type A [26j. But this was not the end of the story. Some ten years later, Kirillov 
and Reshetikhin (TB] discovered an interpretation of Kostka-Foulkes polynomials in 
statistical physics, as generating functions of Bethe ansatz configurations for some 
generalizations of Heisenberg's XXX-magnet model, and obtained a closed expres- 
sion in the form of a sum of products of g-binomial coefficients. 

All of these results have been generalized in many directions. Generalized Hall al- 
gebras (associated with quivers) have been introduced [36]. Ribbon tableaux [22] and 
A;-Schur functions [I9] give rise to generalizations of the charge polynomials, some- 
times interpretable as Kazhdan-Lusztig polynomials [23] . Intersection homology has 
been computed for other varieties. The Kirillov-Reshetikhin formula is now included 
in a vast corpus of fermionic formulas, available for a large number of models jllj . 
However, the relations -if any- between these theories are generally unknown. 

The present article is devoted to a different kind of generalization of the Hall- 
Littlewood theory. It is by now well-known that many aspects of the theory of 
symmetric functions can be lifted to Noncommutative symmetric functions, or quasi- 
symmetric functions, and that those points which do not have a good analogue at this 
level can sometimes be explained by lifting them to more complicated combinatorial 
Hopf algebra^j. The paradigm here is the Littlewood-Richardson rule, which becomes 
trivial in the algebra of Free symmetric functions, all the difficulty having been diluted 
in the definition of the algebra [3] . 

A theory of noncommutative and quasi-symmetric Hall-Littlewood functions has 
been worked out by Hivert |T2j. Since there is no Hall algebra to use as a starting 
point, Hivert 's choice was to imitate Littlewood's definition, which can be reformu- 
lated in terms of an action of the affine Hecke algebra on polynomials. By replacing 
the usual action by a quasi-symmetrizing one, Hivert obtained interesting bases, be- 
having in much the same way as the original ones, and were easily deformable with a 
second parameter, so that analogues of Macdonald's functions could also be defined 
|13j . See also the interesting work of Bergeron and Zabrocki ^. 

However, Hivert's analogues of the Kostka-Foulkes polynomials are just Kostka- 
Foulkes monomials, i.e., powers of g, given moreover by a simple explicit formula. So 
the combinatorial connections to tableaux, geometry and statistical physics do not 
show up in this theory. 

More recently, new possibilities arose with Tevlin's [39] discovery of a plausible 
analogue of monomial symmetric functions on the noncommutative side. Tevlin's 
constructions are incompatible with the Hopf structure (his monomial functions are 
not dual to products of complete functions in any reasonable sense), so it seemed 
unlikely that they could lead to interesting combinatorics. Nevertheless, Tevlin com- 
puted analogues of the Kostka matrices in his setting, and conjectured that they had 



There is no general agreement on the precise definition of a combinatorial Hopf algebra, see [T] 
and [50] for attempts at making this concept precise, and [221 [311 ISSj for more examples. 
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nonnegative integer coefficients. This conjecture was proved in |l14J, and turned out to 
be more interesting than expected. The proof required the use of larger combinatorial 
Hopf algebras, and led to a vast generalization of the Gennocchi numbers. 

In this paper we give a new generalization of Hall-Littlewood functions, starting 
from Tevlin's bases. We define g-analogues S\q) of the products of complete homo- 
geneous functions by embedding Sym in an associative deformation of WQSym and 
projecting back to Sym by the map introduced in [T^. This defines a nonassociative 
g-product -kg on Sym, and our Hall-Littlewood functions are equal to the products 
(see Section IXSll 



(1) S\q) = S'^ ^, iS'^ *,(... iS'^-^ ^g S'^))). 

These functions can be regarded as interpolating between the S^ (at q = I) and a new 
kind of noncommutative Schur functions (at g = 0), have nonnegative coefficients, 
which can be expressed in closed form as products of g-binomial coefficients, and 
have a transparent combinatorial interpretation. As a consequence, the basis Ri{q), 
defined by Moebius inversion on the composition lattice, is also nonnegative on the 
same basis. 

The really interesting phenomenon occurs with Tevlin's second basis (denoted here 
and in \1A\ by Lj), an analogue of Gessel's fundamental basis Fj. One can observe 
that the last column of the matrix M{S, L) (which expresses 5*^" in terms of the 
L/'s) gives the enumeration of permutation tableaux [3H] by shape. This observation 
can be easily shown, and one may wonder whether the expansion of S^"{q) on Lj 
gives rise to interesting g-analogues. This is clearly not the case (there are negative 
coefficients), but it turns out that introducing a simple g-analogue Li{q) of L/, we ob- 
tain again nonnegative polynomials in the matrix M{S{q), L{q)). Finally, the matrix 
M{R{q),L{q)) gives the g-enumeration of permutation tableaux according to shape 
and rank, and another (yet unknown) statistic. Other (conjectural) combinatorial 
interpretations in terms of permutations or packed words are also proposeco. 

As permutation tableaux occur in geometry (they are a distinguished subset of 
Postnikov's J-diagrams, which parameterize cells in the totally non-negative part of 
the Grassmannian [33]) and the g-enumeration of permutation tableaux is (up to 
a shift) counting cells according to dimension. Additionally, permutation tableaux 
occur in physics - Corteel and Williams [4J found a close connection to a well-known 
model from statistical physics called the partially asymmetric exclusion process, 
which in turn is related to the Hamiltonian of the XXZ quantum spin chain [6]. 
Therefore we may say that our new Hall-Littlewood functions have some of the fea- 
tures which were absent from Hivert's theory. However, we do not have the algebraic 
side coming from affine Hecke algebras, and it is an open question whether both 
points of view can be unified. 

We conclude this paper with exact formulas for the g-enumeration of permutation 
tableaux of types A and B, according to shape. In the type A case, by the result of 



Since many of our results and conjectures are stated in terms of permutations, one might be 
tempted to work only with FQSym, bypassing WQSym entirely. However, as was already clear 
in [13], one cannot make sense of the definition of Tevlin's monomial basis ^ using FQSym. 
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Corteel and Williams [1], this gives an exact formula for the steady state probability 
of each state of the partially asymmetric exclusion process (with arbitrary q and 
a = /3 = 7 = (5 = l). Applying results of [3H], this also gives an exact formula for 
the number of permutations with a fixed weak excedance set enumerated according 
to crossings, and for the number of permutations with a fixed set of descent bottoms, 
enumerated according to occurrences of the pattern 2 — 31. 

Acknowledgments.- This work has been partially supported by Agence Nationale de la Recherche, 
grant ANR-06-BLAN-0380. The authors would also like to thank the contributors of the MuPAD 
project, and especially those of the conibinat package, for providing the development environment 
for this research (see |15j for an introduction to MuPAD-Combinat). 



2. Notations and background 

2.1. Words, permutations, and compositions. We assume that the reader is 
familiar with the standard notations of the theory of noncommutative symmetric 
functions [HI E]- We shall need an infinite totally ordered alphabet A = {oi < 02 < 
• • • < a„ < ■ • • }, generally assumed to be the set of positive integers. We denote by 
IK a field of characteristic 0, and by ]K{A) the free associative algebra over A when A 
is finite, and the projective limit proj liin^K{B) , where B runs over finite subsets of 
A, when A is infinite. The evaluation ev(w) of a word w is the sequence whose i-th 
term is the number of times the letter Oj occurs in w. The standardized word Std{w) 
of a word w ^ A* is the permutation obtained by iteratively scanning w from left 
to right, and labelling 1,2,... the occurrences of its smallest letter, then numbering 
the occurrences of the next one, and so on. For example, Std{bbacab) = 341625. 
For a word w on the alphabet {1,2, . . .}, we denote by w[k] the word obtained by 
replacing each letter i by the integer i + k. If m and v are two words, with u of 
length k, one defines the shifted concatenation u»v = u-{v[k]) and the shifted shuffle 
n lyj f = uLLI (f [A;]), where 111 is the usual shuffle product. 

Recall that a permutation a admits a descent at position i if a{i) > a{i + 1). 
Symmetrically, a admits a recoil at i if cr~^(z) > a~^{i + 1). The descent and recoil 
sets of a are the positions of the descents and recoils, respectively. 

A composition of an integer n is a sequence / = (zi, . . . , v) of positive integers of 
sum n. In this case we write I \= n. The integer r is called the length of the compo- 
sition. The descent set of I is Des(/) = {ii, ii + i2, ■ ■ ■ ,ii + ■ — h V-i}- The reverse 
refinement order, denoted by ^, on compositions is such that / = (zi, . . . , z^) ^ J = 
(ii) • • • ) ji) iff Des(J) 3 Des( J), or equivalently, {ii, ii + i2, . . . ,ii + ■ ■ ■ + ik} contains 
{ji,ji +J2, ■ ■ ■ ,ji + ■ ■ ■+ji}- In this case, we say that / is finer than J. For example, 
(2, 1, 2, 3, 1, 2) >z (3, 2, 6). The descent composition DC(o") of a permutation a G (3„ 
is the composition J of n whose descent set is the descent set of a. Similarly we can 
define recoil compositions. 

If / = {ii, . . . ,iq) and J = (ji, . . . , jp) are two compositions, then I- J refers to their 
concatenation (ii, . . . ,iq,ji, . . . ,jp), and I>J is equal to (ii, . . . ,iq + ji,J2, ■ ■ ■ ,jp)- 

The major index maj(-ft') of a composition K = {ki, . . . , kr) is equal to the dot 
product of {ki, . . . ,kr) with (r — 1, r — 2, . . . , 2, 1, 0), i.e. X]I=i(^ ~ O^j- 
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2.2. Word Quasi-symmetric functions: WQSym. Let w G A*. The packed 
word u = pack(w) associated with w is obtained by the following process. If bi < 
b2 < ■ ■ ■ < K are the letters occuring in w, m is the image of w by the homomorphism 
hi H- > Qi. A word u is said to be packed if pack(M) = u. We denote by PW the set of 
packed words. With such a word, we associate the polynomial 

(2) M, := Yl ^- 

pack(w))=u 

For example, restricting A to the first five integers, 

(3) Mi3i32 = 13132 + 14142 + 14143 + 24243 + 15152 + 15153 + 25253 + 15154 + 25254 + 35354. 

Under the abelianization x '■ IK(y4) -^ IK[X], the M^j are mapped to the monomial 
quasi-symmetric functions M/ (/ = (|M|a)agA being the evaluation vector of u). 

These polynomials span a subalgebra of K(y4), called WQSym for Word Quasi- 
Symmetric functions [12j. These are the invariants of the noncommutative version of 
Hivert's quasi-symmetrizing action [12], which is defined hj a ■ w = w' where w' is 
such that Std(w') = Std(w) and x(^0 = o" ■ x(^)- Thus, two words are in the same 
(5(y4)-orbit iff they have the same packed word. 

The graded dimension of WQSym is the sequence of ordered Bell numbers ( |37l 
A000670]) 1, 1, 3, 13, 75, 541, 4683, 47293, 545835, .... Hence, WQSym is much larger 
than Sym, which can be embedded in it in various ways [30l [31] . 

The product of the M^ of WQSym is given by 

(4) M„,M,. = ^ M, , 

where the convolution u'*wu" of two packed words is defined as 

(5) u'*wu" = ^ u. 

v,w,u=v-w G PW,pack(r))=M',pack(ui)=u" 

For example, 

(6) M11M21 = M1121 + M1132 + M2221 + M2231 + M3321. 

M21M121 = M12121 + M12131 + M12232 + M12343 + M13121 + M13232 + Mi3242 
(7) 

+ M14232 + M23121 + M23131 + M23141 + M24131 + M34121. 

2.3. Matrix quasi-symmetric functions: MQSym. This algebra is introduced in 
[121 E]- We start from a totally ordered set of commutative variables X = {xi < ■ ■ ■ < 
Xn} and consider the ideal ]K[X]+ of polynomials without constant term. We denote 
by ]K{X} = T(K[X]+) its tensor algebra. We will also consider tensor products of 
elements of this algebra. To avoid confusion, we denote by "■" the product of the 
tensor algebra and call it the dot product. We reserve the notation ® for the external 
tensor product. 

A natural basis of K{X} is formed by dot products of nonconstant monomials 
(called multiwords in the sequel), which can be represented by nonnegative integer 
matrices M = {rriij), where rriij is the exponent of the variable Xi in the jth factor 
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of the tensor product. Since constant monomials are not allowed, such matrices have 
no zero column. We say that they are horizontally packed. A multiword m can be 
encoded in the following way. Let V be the support of m, that is, the set of those 
variables Xi such that the ith row of M is non zero, and let P be the matrix obtained 
from M by removing the null rows. We set m = V^. A matrix such as P, without 
zero rows or columns, is said to be packed. 



For example the multiword m = a- ab^e^ ■ a?d is encoded by 



"1 12" 


030 


000 


001 


.0 5 0. 



Its support is the set 



{a, h, d, e}, and the associated packed matrix is 



112 
030 
001 
050 



Let MQSym(X) be the linear subspace of K{X} spanned by the elements 

MS, - ^ - ^^^ 



'M 






where Vk{X) is the set of fc-element subsets of X, and M runs over packed matrices 
of height h{m) < n. 



For example, on the alphabet {a <b < c < d) 
MS 

One can show that MQSym is a subalgebra of ]K{X}. Actually, 

MSpMS( 





a 


"1 12' 


a 


"1 12' 


a 


"1 12' 


a 


"000" 


"1 12' 


_ b 
c 


030 
001 


c 


030 
000 


c 


000 
030 


c 


112 
030 


030 
.001. 


d 


000 


d 


001 


d 


001 


d 


001 



E MS 

i?eLij(P,Q) 



R 



where the augmented shuffle of P and Q, LJJ (P, Q) is defined as follows: let r be an 

integer between max(p, g) and p + q, where p = h{P) and q = h{Q). Insert null 

rows in the matrices P and Q so as to form matrices P and Q of height r. Let R be 

the matrix {P,Q). The set Lij(P, Q) is formed by all the matrices without null rows 

obtained in this way. 
For example : 



MS 



io]MS[3i] 



MS 



2 100 
1000 
0031 



MS[Ji°;']+MS 



2100 
0031 
1000 



-MS[?J3i]+MS 



0031 
2100 
1000 



2.4. Free quasi-symmetric functions: FQSym. The Hopf algebra FQSym is 
the subalgebra of WQSym spanned by the polynomials [5] 



(9) 



G. 



Std(u)= 



M. 



E 

Std(to)= 



W. 
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The multiplication rule is, for a E &k and /?€©/, 

(10) G„G^= Yl ^-r- 

Std{u) = a,Std(i>)=/3 

As a Hopf algebra, FQSym is self-dual. The scalar product materializing this duality 
is the one for which (Go-, G^) = S^^t—i (Kronecker symbol). Hence, F^ := Gq-i is 
the dual basis of G. Their product is given by 

(11) F«F^ = Yl ^7- 

■y£aWf3 

2.5. Embeddings. 

2.5.1. Sym into MQSym. Recall that the algebra of noncommutative symmetric 
functions is the free associative algebra Sym = C(S'i, S2, ■ ■ ■) generated by an in- 
finite sequence of non-commutative indeterminates Sk, called complete symmetric 
functions. For a composition I = {ii, . . . ,ir), one sets S^ = Si^^ . . . Si^. The family 
(S^) is a linear basis of Sym. A useful realization, denoted by Sym(y4), can be 
obtained by taking an infinite alphabet A = {ai, 02, ... } and defining its complete 
homogeneous symmetric functions by the generating function 

5^r^„(A) = (1 - ta,)-\l - ta^r' .... 

n>0 

Given a packed matrix P, the vector of its column sums will be denoted by Col(P). 
The algebra morphism defined on generators by 

(12) P: Sr^^ Yl ^^P 

Col(P)=(n) 

is an embedding of Hopf algebras ^. By definition of MQSym, for an arbitrary 
composition, we have 

(13) (3{S^) = Y ^^P- 

Col(P)=7 

2.5.2. Sym into WQSym. The algebra morphism defined on generators by 

(14) a: Sn^ Y ^- 

Std(u)=12---n 

is also an embedding of Hopf algebras. For an arbitrary composition, 

(15) a{S')= Y M"- 

DC(n)^/ 

Indeed, when Sym is realized as Sym(y4), the latter sum is equal to 5*^(^4). 

2.6. Epimorphisms. We shall also need to project back from the algebras MQSym 
and WQSym to Sym. The crucial projection is the one associated with the (non- 
Hopf) quotient of WQSym introduced in fHj . 
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2.6.1. MQSym to WQSym. To a packed matrix M, one associates a packed word 
w{M) as follows. Read the entries of M columnwise, from top to bottom and left to 
right. The word w{M) is obtained by repeating rriij times each row index i. 
Let J' be the ideal of MQSym generated by the differences 

(16) {MSp-MSQ\wiP) = wiQ)}. 

Then the quotient MQSym/ JT is isomorphic as an algebra to WQSym, via the 

identification MSa/ = M^(a/). More precisely, r] : MS a/ i-^ W\.w{m) is a morphism of 
algebras. 



2.6.2. WQSym to Sym. Let w be a packed word. The Word composition (W-comp- 
osition) of w is the composition whose descent set is given by the positions of the last 
occurrences of each letter in w. 
For example, 

(17) WC(1543421323) = (2, 3, 2, 2, 1). 

Indeed, the descent set is {2, 5, 7, 9, 10} since the last 5 is in position 2, the last 4 is in position 5, 
the last 1 is in position 7, the last 2 is in position 9, and the last 3 is in position 10. 

The following tables group the packed words in PW2 and PW3 according to their W-composition. 



2 


11 


11 


12 
21 



3 


21 


12 


111 


111 


112 


122 


123 




121 


211 


132 




212 




213 




221 




231 
312 
321 



(18) 



Let ~ be the equivalence relation on packed words defined by -u ~ t> iff WC(m) = 
WC(f ). Let JT' be the subspace of WQSym spanned by the differences 

(19) {M^-M,\ur^v}. 

Then, it has been shown [13] that J'' is a two-sided ideal of WQSym, and that 
the quotient T' defined by T' = WQSym/jT' is isomorphic to Sym as an algebra. 

More precisely, recall that \l/„ is a noncommutative power sum of the first kind ^. 
Tevlin defined the noncommutative monomial symmetric functions "^j [39J as quaside- 
terminants in the \l/n's. We do not need the precise definition of \E'/ here, only the 
following result. 

Proposition 2.1. [T^ ( : M„ 1-^ ^wc(u) is a morphism of algebras. 

3. Quantizations and noncommutative Hall-Littlewood functions 

In this section, we introduce a new g-analogue S^{q) of the basis S^ of Sym, giving 
two different but equivalent definitions. When we examine the transition matrices 
between this new basis and other bases, we will see a connection to permutation 
tableaux and hence to the asymmetric exclusion process. The new basis elements 
S^{q) play the role of the classical Hall-Littlewood Q'^ [27i Ex. 7. (a) p. 234], and of 
Hivert's Hf{q). 
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3.1. The special inversion statistic. Let m = mi ■ ■ ■«„ be a packed word. We say 
that an inversion Ui = b > Uj = a (where i < j and a < 6) is special if Uj is the 
rightmost occurence of a in u. Let sinv(M) denote the number of special inversions in 
u. Note that if m is a permutation, this coincides with its ordinary inversion number. 

3.2. Quantizing WQSym. Let M^ = g^^^^^^M^ and define a linear map (pq by 
0g(Mu) = M^. We define a new associative product -kg on WQSym by requiring 
that 

(20) M:,^,M; = 0,(M„M,). 

For example, by ([6]), one has 

M'n *, M'21 = M'ii2i + M;i32 + M'2221 + M^23i + M^32i 

= qMii21 + gMii32 + q^M2221 + '7^M2231 + g^M3321. 

This algebra structure on the vector space WQSym will be denoted by WQSym^. 

3.3. Quantizing MQSym. Similarly, the g-product -kg can be defined on MQSym, 
by requiring that the MS'^j = ^"^'"''("'(^■^^^MSm muhiply as the MSm- 

3.4. Two equivalent definitions of S^{q). Embedding Sym into MQSym and 
projecting back to Sym, we define g-analogues of the products S^ by 

(22) S\q)=Cor^{(3{S,,)^g---^g(3{S,,^)). 

Equivalently, since under the above embeddings, the image of Sym in MQSym is 
contained in the image of WQSym, one can embed Sym into WQSym and project 
back to Sym, which yields 

(23) S\q)=aaiS,J^g---^gaiS,^)). 

3.5. The transition matrix M{S{q),'$). For any two bases F, G of Sym, we 
denote by Mn{F,G) the matrix indexed by compositions of n, whose entry in row 
/ and column J is the coefficient of Gj in the G-expansion of Fj. We will give 
two combinatorial formulas (Propositions 13.11 and 13.21) and one recursive formula 
(Theorem 13.61) for the elements of the transition matrix M{S{q), \1/), where the \l//'s 
are Tevlin's noncommutative monomial symmetric functions. 

3.5.1. First examples. Let [n] denote the g-analogue l + g + - ■ -H-g""^ of n. The first 
transition matrices SPn = Mn{S{q), \l/) are 



SPs = Ms{S{q),^) 



/[I] [1] [1] [1] \ 

[1] [3] [2] [2] [2] 

[1] [1] [2] [2] 

Vil] [3] [3] [2][3]y 



10 
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SPa 



/[I] [1] [1] [1] [1] [1] [1] [1] \ 

[1] [4] [3] [2] [3] [2] [2] [3] [2] [2] [2] [2] [2] 

[1] [1] [3] [3] [2] [2] [2] [2] [2] [2] 

[1] [4] [4][3]/[2] [3] [4] [3] [3] [3] [3] [3] [2] [3] [3] 

[1] [1] [1] [1] [2] [2] [2] [2] 

[1] [4] [3] [2] [3] [3] [3] [3] [2] [3] [2] [2] [3] 

[1] [1] [3] [3] [3] [3] [2] [3] [2] [3] 

V[l] [4] [4][3]/[2] [3] [4] [4] [3] [4] [3] [4] [2] [3] [4]/ 



The coefficient of \&/ in S'^{q) will be denoted by Cj{q). 



3.5.2. Combinatorial interpretations. Recall that by Proposition 12. ![ C, orj is a. mor- 
phism of algebras sending MSa/ to \E'wc(w(Af))- Hence, our ffist definition of S'^{q) 
gives the following: 

Proposition 3.1. Let I = {ii, . . . , z^) and J = (ji, . . . ,ji) be two compositions. Let 
M{I, J) be the set of integer matrices M = {fnp,q)i<p<i;i<q<k without null rows such 
that 



(24) 


WC(w(M)) = / and Col(M) = 


= J. 


Then 






(25) 


AfGAf(/,J) 





For example, the six matrices corresponding to the coefficient [4][3]/[2] of M^ in row (2, 1, 1) and 
column (2, 2) are 

(26) 

The corresponding statistics are 

(27) 



'2 ■ 




'll' 




'l l' 




'■ l' 




'■ l' 




'■ l' 


■ 1 




1 ■ 




■ 1 




2 ■ 




1 1 




■ 1 


■ 1 




■ 1 




1 ■ 




■ 1 




1 ■ 




2 ■ 



{0, 1, 2, 2, 3, 4}. 



The second definition of S"^ {q) yields a different combinatorial description: 

Proposition 3.2. Let I and J be two compositions and let W{I, J) be the set of 

packed words w such that 

(28) WC{w) = I and DC{w) ^ J. 
Then 

(29) C/(g) = Yl q"'"""^""^- 

weW{I,J) 

For example, the six packed words corresponding to the coefficient [4] [3]/ [2] of M4 in row / = 
(2, 1, 1) and cohmm J = (2, 2) are 

(30) 1123, 1213, 1312, 2213, 2312, 3312. 

These words are the column readings of the six matrices from (1^^ . The first word has descent 
composition (4) and the others have descent composition (2, 2). 
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3.5.3. The q-product on Sym. To explain the factorization of the coefficients of the 
transition matrix M(S'(g), \1/), we need a recursive formula for S'^{q). This will be 
given in Theorem 13.61 
In WQSym, let 

(31) Sn = «(5„) =J2M^ 

where m | n means that m is a nondecreasing packed word of length n, and define 

(32) S' = S,, ^,...^, 4 

so that S'^{q) = ({S'^). Let J = (ji,...,>) and set J' = (j2, •••,>)• Since -kg is 
associative in WQSyrrig, we have 

(33) S'^'J\q)=CiS^^'J')= J2 9""'^'^C(M„^,M,). 

DC{u)^j' 

This expression can be simplified by means of the following Lemma. 
Lemma 3.3. Let u be a nondecreasing packed word. Then 

(34) C(M,^,M„)=C(M„^,M„0 

for all v' such that WC{v') = WC{v). 

Proof - Since u is nondecreasing, each packed word z = x ■ y appearing in the 
expansion of M^j -kg M^ is completely determined by the letters used in x and the 
letters used in y. Looking at the packed words z and z' occuring in M„ -kg M„ 
and in M„ -kg M^/ with given letters used for their prefixes and suffixes, we have 
sinvf^;') = smv(z) + sinvft;') — sinvff), whence the result. ■ 



' 5 



For example, 

(35) Mil *q Mi2 = M1112 + M1123 + g^M2212 + g^M2213 + (Z'*M33i2. 

(36) Mil *q qM2i = gMii2i + gMii32 + g^M222i + Q^M223i + g'^M332i. 

Let now a : Sym -^ WQSym be the section of the projection ( defined by 

(37) a(^/) = MiH2>2....... 

We can define a (non-associative!) g-product on Sym by 

(38) f^,g = a^if)^,cjig)). 

Then Lemma 13.31 implies that 

(39) S\q) = S'^ ^g {S'^ *,(... {S'-~^ ^g S'^))). 
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3.5.4. Closed form for the coefficients. ^From Lemma [33| we now have 

(40) S^^"''{q)=aS^^''')= Y. C'4(.)(g)C(M.*,Mj. 

"Ti2H ^3r 

Note that (^(M„^g M„) and C(M.u' ^g M^/) are hnear combinations of disjoint sets of 
"^K as soon as the nondecreasing words v and v' are different. So the computation 
of the coefficient Cj boils down to the evaluation of 

(41) Yl C(M„^, M,) = C{S'' ^, M,), 

where u is a nondecreasing word. Let us first characterize the terms of the product 
yielding a given "^j. 

Lemma 3.4. Let u be a nondecreasing word of length k over [l,r]. Given a com- 
position I = {ii, . . . ,ir) of length r, there exists at most one nondecreasing word v 
over [l,r] such that uv is packed and WC{uv) = I. Such a v exists precisely when 
u = ui ■ ■ ■ Uk satisfies Ui < Mj+i for i G Des(/). 

In this case, let y = V^T'^ . . . r'''. Then sinv(Mw) is equal to 

(42) J](n,-2/,). 

l<i<fc 

This sum is also equal to 

(43) Yl «.-(^ + maj(¥)), 

l<i<fc 

where K is the composition of k such that Des(-ft') = Des(/) fl [1, fc — 1]. 

Proof - The construction of v was already given in the proof of Theorem 6.1 of J14j . 
It comes essentially from the facts that the letters which should be used in v are 
determined by the letters used in u, and that a word is uniquely determined by its 
packed word and its alphabet. 

Now, for each letter x of m, its contribution to sinv(Mf ) is given by the number of 
different letters strictly smaller than x appearing in v. This is equal to Ui — yi- The 
sum of the yi is k + maj(i^). h 



For example, given / = 1221, there are 10 nondecreasing words u of [1,4] of length 3 satisfying 
the conditions of the lemma. The following table gives the corresponding v and the sinv statistics 
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of the products uv. 



(44) 



u 


V 


sinv 


122 


334 





123 


224 


1 


124 


223 


2 


133 


224 


2 


134 


223 


3 


144 


223 


4 


233 


114 


3 


234 


113 


4 


244 


113 


5 


344 


112 


6 



We are now in a position to compute 

(45) C(^''^ *, M,) 

when f is a nondecreasing word. 

Lemma 3.5. Let I he a composition of k + n and let I' be the composition of n such 
that Des(/') = {ai, . . . , a^} satisfies {k + ai, . . . ,k + a^} = Des(/) r\[k + l,k + n]. 

Let V be the nondecreasing word of evaluation I' . The coefficient of \1// in ({Sk -^q 
M^) is the q-binomial coefficient 



(46) 

where r 
and s = 

Proof 

(47) 



k + r — s 
r — s 



= /(/), K is the composition of k such that Des{K) 
1{K). 



Des(/) n [1,A;- 1], 



To start with, write 



5fc*,M, = 5^g^^-(-)M, 



where w runs over packed words of the form w = u'v', with u' nondecreasing and 
pack(f') = V. From Lemma I3.4[ we see that in order to have WC{u'v') = I, 
u' = Xi---Xk must be a word over the interval [l,r] with equahties Xj = Xj al- 
lowed precisely when cells i and j are in the same row of the diagram of K. The 
commutative image of the formal sum of such words, which are the nondecreasing 
reorderings of the quasi-ribbons of shape K [T7], is the quasi-symmetric quasi-ribbon 
polynomial FK{ti, . . . ,tr), introduced in [9]. Hence, the coefficient of \E'/ is 



(48) q 

given the generating function |10 

(49) 



-ma.}(K) 



k + r — S 
r — s 



5^t'"F^(l,g,...,g 



m— 1^ 



m>0 



{t;q) 



k+1 
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The example presented in (|44|) corresponds to the case I — (1,2,2, 1) and i = 3, so that K = (1, 2) 
[3 + 4-2] 
4-2 



We then find 



which indeed corresponds to the statistic in the last column 



of (III 



Summarizing the above discussion, we can now state the main resuh of this section: 

Theorem 3.6. Let I = (zi, . . . ,ik) and J = (ji, . . . ,ji) be compositions of n. Then 
the coefficient Cj{q) of ^ i in S'^{q) is given by the following rule: 
(i) «/h < Ji, then Cf{q) = C(l+,2,i3,...,,,)(g), 
(a) otherwise, 



(50) 



chq) 



k + ji-l 
h 



^iJ2,---,jl) 



Cf-''>{q) 



where the diagram of I' is obtained by removing the first ji cells of the diagram of I. 



3.6. The transition matrix M(i?(g), \&). 

3.6.1. q-deformed ribbons. We now define a g-ribbon basis Ri{q) in terms of the 
S'^{q)'s by analogy to the relationship between the ordinary Rj's and S'^'s: 



(51) 



RM ■■= $:(-i)'(^)-'(^)^^(g). 



j-</ 



The coefficient of "^j in the expansion of R^ {q) will be denoted by Dj{q). 

3.6.2. First examples. We get the following transition matrices between R{q) and \1/ 
for n = 3, 4: 



RP, = M,(R, *) 



(I . . .\ 

I q + q^ q 

1 . q . 

\l q + q^ q + q^ q^ J 



RPa 



[\ . . . . 

1 g[3] g+g^ . q 

1 . g+g^ . g 

1 g[3] g+2g2 + g3 + g4 ^3^3] ^.^^2 

1 . . . g 

1 g[3] g+g^ . g+g^ 



g^+g^+g^ 



g^+g^+g^ 



g,o 



\ 



1 . g+g . g+g . g 

Vl g[3] g+2g2 + g3 + g4 gS^] ^[3] g2^^3^2g4 + g5 g3[3] gSy 
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3.6.3. Combinatorial interpretations. By definition of tlie transition matrix from S{q) 
to R{q), the matrices M{R{q), \1/) can be described as follows: 

Proposition 3.7. Let I and J be compositions of n, and let W'{I,J) be the set of 
packed words w such that 

(52) WC{w) = I and DC{w) = J. 

Then 

^sinv(ui) 
w&W'{I,J) 



(53) Diiq)= Yl r 



Proof - This follows directly from the combinatorial interpretation of C/ in terms of 
packed words (see Proposition] 



In terms of MQSym, this can be rewritten as follows: 

Corollary 3.8. Let I and J be compositions ofn. Then Dj{q) is given by the statistic 
sinv(w(M)) applied to the elements M of the subset of M{I,J) where in each pair 
of consecutive columns, the bottommost nonzero entry of the left one is strictly below 
the top-most nonzero entry of the right one. 

3.7. The transition matrix M(L(g),\I'). 

3.7.1. A new q-analogue of the L basis o/Sym. Let st(/, J) be the statistic on pairs 
of compositions of the same weight defined by 

(54) st(/ J) ■= I #^(''^') ^ °^'(^) "" Bes{J)\t > j} my J, 

^ ^ I ' J • I _gQ otherwise 

We define a new basis L{q) by 

(55) Lj{q) := J^ <l''^'''^^i = E <l''^' ''^ "^ i ■ 

i\=\J\ i^J 

For g = 1, this reduces to Tevlin's basis L/ (in the notation of [H]). Since M{Lj{q), \E') 
is unitriangular, Lj{q) is a basis of Sym. 

3.7.2. First examples. Here are the first transition matrices from L{q) to \E': 



(56) MLPs = MsiLiq),^) 



(I 






\ 


1 


q 






1 




q 




V 


Q 


q' 


q^j 
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(57) 



MLP4 



/I 

1 

1 
1 
1 

1 

1 

\1 



• Q ■ 
q q q 



Q 



Q 






qo 



q q^ q^ q^ q^ q^ q^/ 

Note that up to some minor changes (conjugation w.r.t. mirror image of compo- 
sitions), these are the matrices expressing Hivert's Hall-Littlewood Hj on the basis 
Rj p^. This allows us to derive the expression of their inverse, that is, transition 
matrices from \E' to L[q) (see [12], Theorem 6.6): 



(58) 

where st'(/, J) is 
(59) st'(/, J) : = 



^j = 5^(-l/g)'«-'(^)g-*'(^-^)L,(g), 



lyj 



#{(«, j) G Des(/) X Des(J)K < j} if / ^ J, 
—00 otherwise 



3.8. The transition matrix AI{S{q), L{q)). The coefficient oi Lj{q) in S'^{q) will be 
denoted by Ej{q). In this section, we will see a connection to permutation tableaux 
and hence to the asymmetric exclusion process. 

3.8.1. First examples. Here are the first transition matrices from S{q) to L{q): 



(\ 



SLs = Ms{S{q),L{q)) 



V- 



1 1 1 \ 

1+q 1 2+g 
1 1 
1 



/I 



SL4 



1 

l + q+q^ 



1 

l + q 
l + q 



1 

2 + 2g + g2 

1 + g 
1 + 2g + g2 



2 + 2g+g2 

1 
l+q 

1 
l + q 



I 
2 + q 
2 + q 
l + q 

I 

I 

1 



1 \ 

3 + 3g + g2 

2 + q 
3 + 3g + g2 

1 
2 + q 
I 
1 / 



In fact the right-hand column of each of these matrices contains the (un-normalized) 
steady-state probabilities of each state of the partially asymmetric exclusion process 
(PASEP). More specifically, the steady-state probabilities of the states ••, #0, o«, 
and 00 (the states of the PASEP on 2 sites) are -^, ^^, ^, and ^, respectively; 
compare this with the right-hand column of SL3. The steady-state probabilities of 
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• ••, ••o, •o«, #0 0, o««, o»o, oo», ooo are given by the right-hand column of 
5'L4. This will be proved in Section HI building on work of [4J. 

Note that since all coefficients of the matrix SPn are explicit (and products of 
g-binomials) and since SLn comes from SPn by adding and substracting rows, we 
have a simple expression of Ej{q) as an alternating sum of Cf{q), hence of products 
of g-binomials. 

Note that these matrices are invertible for generic values of q, in particular for 
q = 0. Hence, we can define Hall-Littlewood type functions by 

(60) Hj{q) = Y,Ei{q)Li 

I 

which interpolate between S*^ (at g = 1) and a new kind of noncommutative Schur 
functions S/ at g = 0. 

We will see in the next section that the last column gives the g-enumeration of 
permutation tableaux according to shape. Let us write down the precise statement 
in that case. 

Proposition 3.9. Let ci{q) be the coefficient of ^/ in S^"{q). Let ei{q) be the 
coefficient of Lf{q) in S^"{q). 
Then, 

(61) e,(g) = J2^~l/qy^'^''^'k-''''''''^cj{q), and 

(62) c,„...,,v(g) = [rY,'[r - l]f . . . [^-'llt ■ 

Proof - Straightforward from Theorem 13.61 and Equation [5H1 ■ 

We shall use the notation QFactyi(J) := cj{q) in the sequel, regarding it as a 
generalized g-factorial defined for all compositions (the classical one coming from 

J =(!«).) 

3.8.2. A combinatorial lemma. We need to describe the g-product in the L(g) basis. 
Our first objective will be to understand how st(/, K) can be related to st(/, J) and 
st(J,ir) for all ir^ J^ /. 

Lemma 3.10. Let X = {xi < ■ ■ ■ < Xr} ^ Z = {zi < ■ ■ • < Zm\ be two sets of 
positive integers. For an integer y, let 

(63) uiy) = i^{z E Z\z > y} + #{x G X\x < y}, 
and for a set Y, 

(64) uiY) = J2^iy)- 

yev 
For r < s < m, let 

(65) S,(X,Z)= ^ g'^(^). 



XCYCZ 

\Y\ = s 
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S,(X,Z) = g'^(^)+('-+^)(^-^)+(V) 



m — r 

s — r 



Proof - Let Z/X = U = {ui < ■■ ■ < Um-r} and Ui = v{ui). Then z/j = m — i + 1, so 
that all Vj are consecutive integers. By definition, 



S,(X,Z)= Y. ^ 

XC{y^<-<ys}CZ 



'^{yi)+--+'^{ys) 



(67) 



g^We,.,(g^S...,g'^— ), 



where e„(X) is the usual elementary symmetric function on the alphabet X. Thus, 



S,(X,Z) = g'^We,_,(g^+\...,g™) 



(68) 



^.(X)^(.+l)(.-Oe^_^(l^^^,,,^^m-r-l) 



u{X)+{r+l){s-r)+{^-/) 



m — r 
s — r 



For example, with X = {3, 7} and Z = {1, . . . , 10}, one has 



yez/x 



(70) 



7\ 22 



E4(X, Z) - g^'^e2(g^ . . . , g^") = q^'e^ih ■...q')^q 



3.8.3. r/ie q-product on the basis L{q). Recall that the g-product on Sym is a non- 
associative product but that S^{q) is the g-product on the parts of J, multiplied from 
right to left (see 



Lemma 3.11. For an integer p and a composition I, 
(71) L,(g)^,L,(g)= Y ^''^'"'""'^ 

K^pt>I 

where mx = i^{k G Des{K)\k > p}. 



niK +p 
p 



^ 



K, 



Proof - Note that Lp{q) = Sp. The g-products Sp-kg'^L are easily computed by means 
of Lemma 13.51 ■ 



We are now in a position to expand such g-products on the L(g) basis: 
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(72) M,) *,!,(,) =$:,■'«-"-(«■";"■■)-(■?) 

Jl>p 

Proof - From Lemma [3.111 with r = /(/) — 1, we have 



■/(/)+p-l 
/(J) - 1 



LAq). 



-I Q 



LM)^,LM)= E g^^^^'^^'^J^g^^' 



(73) 



EE 

Kyp[>I s>r 



s>r 



p — r 
p — S 



-I <? 



p + r 

S 



^ 



K 



-I 1 



p + r 
s 



Q 



st(K,p\>I)+rs 



p — r 
p — s 



^ 



K 



Thanks to Lemma EUHl noting that st{K,J) + st{J,pt>I) = u{Des{J)) with X 
Des(/) and Z = Des{K) fl [p, cxd], this is equal to 



E 

s>r 



p + r 

s 



j2 q-^(J'P>n+i'7)-i''V) J2 g^*(^'^)^ir 



(74) 



<? Jhp>i 

l{J) = s + l. Ji>p 



E' 

jyp>i 
ji>p 



st(J,p>/)+('(^)-'(^')-('W) 



'1{I)+P-1 

/(J) - 1 



K>~J 



LAq). 



For example, 

(75) L2{q) •g L2i{q) = [3]i4l(g) + [3]i3ll('Z) + [3]i22l(g) + qL2iii{q). 



(76) 



(77) 



L2{q) *q Li2{q) = [3]L32(<?) + q[3]L3n{q) + [3]L2i2(g) + q^L2iii{q). 



Lsiq) -kg L22{q) =[4]L52+(7 



^511 



^412 +(?^[4]i4iii 



^322(9) + <?^[4]i3211 + 9[4]i3112 + q'^L^iiii. 



Since S'-^(g) = Lj^{q) -kg [LjAq) ^q (• • • {Lj^_Aq) *q Lj^{q)) ...)), Formula [72] imphes 
the following. 

Corollary 3.13. T/ie coefficient Ej{q) is in N[g]. 

Corollary 3.14. Recall that ei{q) is the coefficient of Lj{q) in S^"{q). Then, for 
any composition I = (ii, . . . , v); 



(78) 



ei+n,i^,...,iM = Hge/ + 5^g^ ^ei,,„„i,+i,^^,„„,^(g), 



k=l 



el,^,,i2,...,^riq) =ei{q). 
Conversely, this property and the trivial initial conditions determine completely the 



ei- 
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Proof. This follows from the fact that S^" (q) = Li{q) -kg {. . . (-Li(g) -kg Li{q)) . . .), by 
putting p = 1 into (172|) . 



3.8.4. Towards a combinatorial interpretation of Ej{q) . In Theorem 13. 171 we will give 
a combinatorial interpretation of the coefficients Ej{q) expressing S'^{q) in terms of 
the Lj{q)^s. But first we need a new combinatorial algorithm sending a permutation 
to a composition. 

Let 0" be a permutation in ©„. We compute a composition LC(cr) of n as follows. 

• Consider the Lehmer code of its inverse Lh(o"), that is, the word whose ith 
letter is the number of letters of a to the left of i and greater than i. 

• Fix S* = and read Lh(cr) from right to left. At each step, if the entry k is 
strictly greater than the size of S, add the (A;-#(S))-th element of the sequence 
[l,n] with the elements of 5* removed . 

• The set S is the descent set of a composition C, and LC(cr) is the mirror 
image C" of C. 

For example, with a = (637124985), the Lehmer code of its inverse is Lh(cr) = (331240010). Then 
5* is at first, then the set {1} (second step), then the set {1,4} (fifth step), then the set {1,4,2} 
(eighth step). Hence C is (1, 1, 2, 5), so that LC{a) = (5, 2, 1, 1). 

One can find in Section [7] the permutations of ©3 and ©4 arranged by rows ac- 
cording to their LC statistics and by columns according to their recoil compositions. 

3.8.5. A left Synig-module. Let ~ be the equivalence relation on (5„ defined by cr ~ r 
whenever LC(o-) = LC(r). Let JH be the quotient of FQSym [5J by the subspace 

(79) V = {F, -F,|a~r}. 

For a composition /, set k{I) = maj(/), and for a permutation, K,{a) = K{LC{a)). Let 
J-'i denote the equivalence class of g'^^'^^Fo-. Denote by o^ be the g-product of FQSym 
inherited from WQSym. More precisely, if F'^ = q^'^^^'^^F^ and 0g(Fo-) = F'^, then 

(80) f;o,f; = 0,(f.f.). 

This is the same structure as the one considered in [5]. In particular, in the basis 
Go- = Fo-i, the product is given by the g-convolution 

(81) G«O^G^= Yl giM7)-mv(a)-inv(/3)Q^_ 

'y^U' ■ -v 
Std(u) = a,Std(u)=/3 

Lemma 3.15. The quotient vector space M. is a left Sym-module for the q-product 
of FQSym, that is, 

(82) F = G modV ^ SpOgF = SpOgG mod V . 

Proof - Let a^^ ~ r^^ G Gi and n = p + I. We need to compare the codes of the 
permutations appearing in the g-convolutions 



^3) U = Gi2...p Og G^ and V = Gi2...p o^ G^ . 
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For a subset S = {si < S2 < ■ ■ ■ < Sp} of [n], let as and ts be the elements of U and 
V whose prefix of length p is S1S2 ■ ■ ■ Sp. Then, the codes of as and ts coincide on the 
first p positions, and are equivalent on the last / ones, so that as ~ ts- Moreover, 
as and ts arise with the same power of q, so we have a module for the g-structure as 
well. ■ 



For example, 



F12 Oq 9F132 — gFi2354 + q F13254 + q F13524 + q F13542 + q F31254 



(84) 



q F31524 + q F31542 + q F35124 + q F35142 + q F, 



35412 



2- 



(85) 



= J^41 + 9.^41 + .^311 + -^221 + q J^A\ 
+ qTzw + qT22\ + (^Tzw + 9^.^221 + qTi\\\- 
F12 Og '7F312 = 9F12534 + q F15234 + q F15324 + q F15342 + q F51234 
+ q F51324 + q F51342 + q F53124 + q F53142 + q F53412 
= Fax + 9.^41 + .^311 + -^221 + q ^Ai 



qJ^zii + qT22\ + (^T'ixi + 9^:^221 + qT2\\\- 



F12 o, gF 



(86) 



213 


— '?Fi2435 + 9''Fi4235 


+ q F14325 + q F14352 + 


q F41235 




+ q F41325 + 9''F4i35 


2 + g F43125 + q F43152 + 


q F43512 




= ^^^41 + q^Al + .?^311 


+ .?^221 + q^^Al 






+ qHix + gJ^22i + q^ 


J'zw + q J'22\ + 9.^2111 ■ 





We have now: 
Lemma 3.16. The left q-product of a Ti by a complete function is given by ^ 

'l{I)+p-l 



(87) SpO^T,= J2 q-i^'P>nH^'''-.'n-i'^P) 



Jl>p 



1{J) - 1 






Proof - Let us first show that this is true at g = 1. Let a be such that LC(cr ^) 



By definition of LC, the permutations r occuring in Gi2...p o^ Go- satisfy LC(r ^) ^ 
LC(o"^^), and the codes of those permutations have the form 

(88) SiS2---Sptit2---ti, 

where t = tit2 ■ ■ ■ t; is the code of a and si < S2 < ■ ■ ■ < Sp. The compositions J such 
that 1{J) —1{I) has a fixed value m will all be obtained by fixing the last m values 
Sp, . . . , Sp_m+i in a way depending on the code t, the first p — m being allowed to be 
any weakly increasing sequence 

(89) si < S2 < . . . < Sp < 1{J) — 1 , which leaves I J choices. 

Now, in the g-convolution Gi2...pOg G^, these permutations r occur with a coefficient 
g,mv(T)-mv((T)^ so that the coefficient of JFj is, up to a power of g, the g-binomial 



coefficient 



p + l{I)-l 
1{J) - 1 



By our choice of the normalization JF/ = g'^^'^^Fo-, this 



q 



power of g is the same as in (1721) . 
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As one can check on the previous examples, we have indeed 
(90) T2 oq ^21 = (1 + 9 + g')^4l + {l + q + g')^3ll + (1 + g + q^)^22l + 9^2111. 



By Lemmas 13. 161 and fl72|) . the two bases JF and L{q) have the same muhiphcation 
formula, so that Ej[q) is also the coefficient of JF/ in the expansion of S'^{q). Hence 

Theorem 3.17. Let I and J be two compositions of n. Let PP(/, J) be the set of 

permutations whose LC statistic is I and whose recoil composition is finer than J. 
Then, 



(91) 



o-ePP(/,J) 



3.9. The transition matrix M{R{q),L{q)). The last transition matrix which re- 
mains to be computed is the one from R{q) to L{q). 



3.9.1. First examples. We have the following matrices for n = 3,4: 



RL3 = Ms{R{q),L{q)) 



A . . A 

l + g 1 . 

1 . 

V- • • V 



/I 



RLa 



l + g + g^ 1 + g 



1 q . 

1+q . 1 . . 

q 1 + q + q^ . l + q 1 
1 

. 1 + g 1 
1 



• 1/ 



3.9.2. Combinatorial interpretation. The coefficient of Lj{q) in Rj{q) will be denoted 
by F/(g). 

/^From the characterization in Theorem 13.171 of M{S{q),L{q)) in terms of permu- 
tations we obtain: 

Theorem 3.18. Let I and J be two compositions. Let PP'(/, J) be the set of per- 
mutations whose LC statistic is I and whose recoil composition is J. The coefficient 
Ff of Lj{q) in the expansion of Rj{q) is given by 



(92) 



-maj(LC(a)) 



y^ inv{o-) ^ 



o-£PP'(/,J) 
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4. The PASEP and type A permutation tableaux 

Permutation tableaux (of type A) are certain fillings of Young diagrams with O's 
and I's which are in bijection with permutations (see [3H] for two bijections). They 
are a distinguished subset of Postnikov's (type A) J-diagrams [35], which index cells 
of the totally non-negative part of the Grassmannian. 

Apart from this geometric connection, permutation tableaux are of interest as 
they are closely connected to a model from statistical physics called the partially 
asymmetric exclusion process (PASEP) ^. More precisely, the PASEP with n sites 
is a model in which particles hop back and forth (and in and out) of a one- dimensional 
lattice, such that at most one particle may occupy a given site (the probability of 
hopping left is q times the probability of hopping right.) See [4j for full details. 
Therefore there are 2" possible states of the PASEP. There is a simple bijection from 
a state r of the PASEP to a Young diagram A(r) whose semiperimeter is n + 1. 
The main result of [4J is that the steady state probability that the PASEP is in 
configuration r is equal to the g-enumeration of permutation tableaux of shape A(r) 
divided by the g-enumeration of all permutation tableaux of semiperimeter n + 1. 

In this section we will give an explicit formula for the g-enumeration of permuta- 
tion tableaux of a given shape. So in particular this is an explicit formula for the 
steady state probability of each state of the PASEP. Additionally, by results of [38] , 
this formula counts permutations with a given set of weak excedances according to 
crossings; it also counts permutations with a given set of descent bottoms according 
to occurrences of the pattern 2 — 31. 

4.1. Permutation tableaux. Regard the following {k^n — k) rectangle (here k = 3 
and n = 8) 



(93) 



as a poset Q^ „: the elements of the poset are the boxes, and box b is less than b' if b 
is southwest of b'. We then define a type A Young diagram contained in a {k,n — k) 
rectangle to be an order ideal in the poset Q^^- This corresponds to the French 
notation for representing Young diagrams. We will sometimes refer to such a Young 
diagram by the partition A given by the lengths of the rows of the order ideal. Note 
that we allow partitions to have parts of size 0. 

As in [21], we define a type A permutation tableau T to be a type A Young dia- 
gram Yx together with a filling of the boxes with O's and I's such that the following 
properties hold: 

(1) Each column of the diagram contains at least one 1. 

(2) There is no which has a 1 below it in the same column and a 1 to its left in 
the same row. 



24 



J.-C. NOVELLI, J.-Y. THIBON, AND L. K. WILLIAMS 



We call such a filling a valid filling of Y\. Here is an example of a type A permutation 
tableau. 



(94) 






1 


1 






1 





1 


1 




1 





1 





1 



Note that if we forget the requirement (1) in the definition of type A permutation 
tableaux then we recover the description of a (type A) J-diagram [3S], an object 
which represents a cell in the totally nonnegative part of a Grassmannian. In that 
case, the total number of I's corresponds to the dimension of the cell. 

We define the ranA; rank(T) of a permutation tableau (of type A) T with k columns 
to be the total number of I's in the filling minus k. (We subtract k since there must 
be at least k I's in a valid filling of a tableau with k columns.) 

4.2. Enumeration of permutation tableaux by shape. Starting from a partition 
with k rows and n — k columns, one encodes it as a composition / = (ii, . . . ,ik) of 
n as follows: Zi — 1 is the number of columns of length k, 12 — 1 is the number of 
columns of length k — 1, and . . . , ifc — 1 is the number of columns of length 1. 

Let i{I) denote the number of parts of /. Then the number PTj^ of permutation 
tableaux of shape corresponding to / is given by a simple formula coming from 
combinatorics of noncommutative symmetric functions. Indeed, according to 
Proposition 9.2], 



(95) 
where 



L-,=Y,9i'^i, 



H=n 



1{I) 



(96) g^ = Y[{iii)-k + iy- 

k=l 

Hence, the coefficient ej of 

(97) L^,=Y,ejLj 

is given by 



J\=n 



1{I) 



(98) 



ej = ^(-l)'«-'(^) Y[{l{I) -k + 1)"-. 



i>j 



k=l 



Moreover, from [14], Theorem 5.1, we known that ej is the number of permutations 
such that GC(cr) = /. Finally, since permutation tableaux of a given shape are in 
bijection with permutations with given descent bottoms [3H] and that GC does the 
same up to reverse complement of the permutations, this number is also the number 
of permutation tableaux of shape /. 
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Theorem 4.1. 

(99) PT^ = 5^(-l)^(^)-^(-^)Fact(J), 



j<i 



where the sum is over the compositions J coarser than I and where Fact is defined 
by 

(100) Fact(ji, . . . ,jp) := p'^p - if' . . . 2^p-n^r 



For example with / = (3, 4, 1), we get 

(101) PT^4i = 3^2^11 - 2^1^ - 2^1^ + 1^ = 297. 

4.2.1. q-enumeration of permutation tableaux according to their shape. In this section, 
we make the connection between the coefficients e/(g) previously seen, and the q- 
enumeration of permutation tableaux. Recall that e/(g) is the coefficient of Li{q) 
in S'^"(g). We saw in Corollary 13.141 that for all compositions / = (ii,...,v), the 
following hold: 

• e(i,ii,i2,...,i^)(g) = ei{q). 

It is possible to transform this result into a g-enumeration of permutation tableaux 
by their rank. Let 

(102) PT^(g) :=^g'-'^"'^(^), 

T 

where the sum is over all permutation tableaux whose shape corresponds to /. 

The following result generalizes Theorem 14.11 Its proof follows directly from Propo- 
sition 13.91 Corollary 13.141 and Lemma 14.51 below. 

Theorem 4.2. Let I be a composition. Then, 

(103) PT^(g) = ej{q) = 5^(-l/g)'(^)-'(^)g-^*'(^'^)QFact^(J), 

where QFact^ is recalled to be 

(104) QFact^(ji, . . . , jp) := [pY^ip - i]f . . . [2yr'[iy^- 

By the results of [4] , Theorem 14.21 gives an explicit formula for the steady state 
probabilities in the partially asymmetric exclusion process (PASEP). More specifi- 
cally, consider the PASEP on a one-dimensional lattice of n sites where particles hop 
right with probability dt, hop left with probability qdt, enter from the left at a rate 
dt, and exit to the right at a rate dt. Let us number the n sites from right to left 
with the numbers 1 through n. Then we have the following result. 

Corollary 4.3. Recall the notation of Theorem \4.S\ Let I be a composition of n + 1, 
and let Zn denote the partition function for the PASEP. Let r denote the state of 
the PASEP in which all sites of Des(/) are occupied by a particle and all sites of 
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[n — 1] \ Des(/) are empty. Then the probability that in the steady state, the PASEP 
is in state t, is 



Z. 



n 



By the results of [3^, this is also an explicit formula enumerating permutations 
with a fixed set of weak excedances according to the number of crossings] equivalently, 
an explicit formula enumerating permutations with a fixed set of descent bottoms 
according to the number of occurences of the generalized pattern 2 — 31. See [38j for 
definitions. 

More specifically, let / be a composition of ra + 1, let DB{I) be the descent set of 
the reverse composition of /, and let W{I) = {1} U {1 + DB{I)}. Here 1 + DB{I) 
denotes the set obtained by adding 1 to each element of DB{I). If a is a permutation, 
let (2 — 31)cr denote the number of occurrences of the pattern 2 — 31 in a, and 
let cr(cr) denote the number of crossings of a. Let T/(g) = Xlo- 9'^^'^^^'^ ^^ ^^^ 
sum over all permutations in Sn+i whose set of descent bottoms is DB{I). And 
let Tj{q) = '^^q'^'^^'^^ be the sum over all permutations in Sn+i whose set of weak 
excedances is W{I). 

Corollary 4.4. 

Ti{q) = T'M) = 5^(-l/g)'«"'(-^)g-*'(^'^)QFact^(J). 

For example with / = (3, 4, 1), the compositions coarser than / are (3, 4, 1), (7, 1), 

(3,5), and (8), so we get 

(105) ^^34l(?)-^2^ ^ ^ ^ +^ 

= g^ + 7q^ + 24g^ + 52g^ + 76g^ + 75g^ + 47g + 15. 

The descent set D{I) of / is {3, 7}, which corresponds to the following state of the 
PASEP: r = «ooo«oo. Therefore the probability that in the steady state, the 

PASEP is in state r, is >7^+7.«+2V+52,^+76,«+75,^+47,+15 ^ 

The polynomial g^ + 7q^ + 24g^ + 52g^ + 76g^ + 75g^ + 47g + 15 also enumerates 
the permutations in Sg with set of descent bottoms {1,5} according to occurrences 
of the pattern 2 — 31. And it enumerates permutations in Sg with weak excedances 
in positions {1, 2, 6} according to crossings. 



The reader might want to compare (I103p with (l6Tll . 
Lemma 4.5. Let I = (ii, . . . ,ir) be a composition. Then 

.A , 

■ (l,li,l2v.v)' 



(106) PTf,,^,^,,..,^)(g) = PT^(g), 



(107) PTfi+,,„...,.)(g) = [r],PT^ + E^'"'PT(l„...,,+,,,„...,.)(g). 



fc=i 
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Proof - First note that PTA^^^^ i^){l) = PTf{q)'- this just says that the g-enum- 
eration of permutation tableaux of shape A is the same as the g-enumeration of 
permutation tableaux of shape A', where A' is obtained from A by adding a row of 
length 0. 

Therefore we just need to prove the second equality. Let A = (Ai, . . . , A,.) be a 
partition. Then, in terms of partitions, the statement translates as: 
fl08) 



PTf(g) 



[r]PTtx, 



-l,A2-l,...,Ar-l) 



(^)+E^'"^^(l 



fc=i 



(Ai,...,A, — k,X, — fc+i,A, — fc+2 — li 



n(«). 



where A^-fc+i means that this part has been removed. 

To this aim, we need to introduce the notion of a restricted zero. We say that a 
zero in a tableau is restricted if there is a 1 below it in the same column. Note that 
every entry to the left of and in the same row as the restricted zero must also be zero. 

We will prove the recurrence by examining the various possibilities for the set 5* of 
r boxes of the Young diagram A which are rightmost in their row. We will partition 
(most of) the permutation tableaux with shape A based on the position of the highest 
restricted zero among S. 

We will label rows of the Young diagram from top to bottom, from 1 to r. Consider 
the set of tableaux obtained via the following procedure: choose a row k for 1 < k < 
r — 1, and fill it entirely with O's. Also fill each box of 5* in row i for any i > k with a 1. 
Now ignore row k and the filled boxes of S, and fill the remaining boxes (which can be 
thought of as boxes of a partition of shape A' := (Ai, . . . , A^-i, A^, A^+i — 1, . . . , A^. — 1)) 
in any way which gives a legitimate permutation tableau of shape A' (see Figure [TJ) 
Note that if we add back the ignored boxes, we will increase the rank of the first 
tableau by A; — 1. So the g-enumeration of the tableaux under consideration is exactly 



EUq'~'pt, 



(Ai 



-fe + 1: 



-fc+2- 



-l,...,Ar-l) 



-n(^)- 



00000000 



Figure 1. 

Let us denote the columns of A which contain a north-east corner of the Young 
diagram A as Ci,...,Ch', we will call them corner columns. Denote the lengths of 
those columns by Ci, . . . ,Ch, so Ci > ■ ■ ■ > Ch- And denote the differences of their 



lengths hy di := Ci — C; 



2, 



.d. 



h-l 



a 



h-l 



Ch. dh 



C, 
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Clearly, our procedure constructs all permutation tableaux of shape A with the 
following description: at least one box of S" is a restricted zero. Furthermore, if we 
choose the restricted zero of S (say in box b) which is in the lowest row (say row k), 
then every box of S* in a row above k is filled with a 1. Equivalently, each corner 
column Cj left of b has its top dj boxes filled with I's, and contains at least dj + 1 ones 
total; and the corner column containing b contains at least d+ 1 ones total, where d 
is the number of boxes above b in the same column. 

The permutation tableaux of shape A which this procedure has not constructed are 
those tableaux such that either no box of S* is a restricted zero, or else there is a box 
of S which is a restricted zero. Let b denote the lowest such box. The condition that 
all boxes of 5* above b must be I's is violated. Let W denote this set of tableaux. 

The following construction gives rise to all permutation tableaux in W (See Figure 
El) Choose a corner column Cj and a number m such that 1 < m < dj. Fill the 



Figure 2. 

top m boxes of Cj with I's and the remaining boxes with O's. For each i < j, fill 
the top di boxes of column Cj with I's. Now ignore the boxes that have been filled, 
and choose any filling of the remaining boxes - which form a partition of shape 
A" := (Ai — 1, A2 — 1, . . . , Ar — 1) - which gives a legitimate permutation tableau of 
shape A". Note that adding back the boxes we had ignored will add di + - ■ ■ + dh-i +m 
to the rank of the tableau of shape A'. Since the quantity di + ■ ■ ■ + d^-i + m can 
range between and r — 1, the rank of the tableaux in W is [r]PT(Ai-i,A2-i,.-,Ar-i)(?)- 



Note that we could give an alternative (direct) proof of Theorem 14.21 by using the 
following recurrences for permutation tableaux (which had been observed in |38j). 
See Figure O for an illustration of the second recurrence. 

Lemma 4.6. The following recurrences for type A permutation tableaux hold. 



prpA 



{il,i2,...,in) 






prpA 



,in) 



(?) 



+ -P^(l,il+i2-l,i3, -,««)(?) ■ 



?^^-l.. + 1.3,....„)(^)+^^-l 



«2v>«n) 



(?) 
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Figure 3. 



5. Permutation tableaux and enumeration formulas in type B 



One can also define [18] type B J-diagrams and permutation tableaux, where the 
Type B J-diagrams index cells in the odd orthogonal Grassmannian, and type B 
permutation tableaux are in bijection with signed permutations. In this section, we 
will enumerate permutation tableaux of type B of a fixed shape, according to rank. 
This formula can be given an interpretation in terms of signed permutations. 

To define type i?„ Young diagrams, regard the following shape 



(109) 



as representing a poset Q^ (here n = 4): the elements of the poset are the boxes, and 
box b is less than b' if b is southwest of b' . We then define a type Bn Young diagram 
to be an order ideal in the poset Q^. 

As in [IS], we define a type B permutation tableau T to be a type B Young dia- 
gram Yx together with a filling of the boxes with O's and I's such that the following 
properties hold: 

(1) Each column of the diagram contains at least one 1. 

(2) There is no which has a 1 below it in the same column and a 1 to its left in 
the same row. 

(3) If a diagonal box contains a 0, every box in that row must contain a 0. 

Here is an example of a type B permutation tableau. 



(110) 



1 












1 











1 


1 




1 
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Note that if we forget requirement (1) in the definition of a type B permutation 
tableaux then we recover the description of a type B J-diagram [18], an object which 
represents a cell in the totally nonnegative part of an odd orthogonal Grassmannian. 

As before, we define the rank rank(T) of a permutation tableau T (of type B) with 
k columns to be the total number of I's in the filling minus k. 

Starting from a type B Young diagram Yx inside a staircase of height n + 1, we 
encode it as a composition of n as follows. If k is the width of the widest row of Yx, 
then I = {ii, . . . , ik+i) is defined by: ii + 1 is the number of rows of length k, 12 is 
the number of rows of length fc — 1, . . . , ik+i is the number of rows of length 0. 

We now explain how to enumerate type B permutation tableaux of a fixed shape 
according to their rank. 

Define QFact^ by 

QFact5(ji, . . .,ip) :=QFact^(ji, . . . , jp) JJ (l + g*) 

(111) *=' 

^ ' p-i 

=[p]f[p-i]f---[2]rMi]?n(i+^*)- 

t=l 
Theorem 5.1. Let I he a composition. 

(112) PTf (g) = 5^(-l/g)'(')-'('^)g-^*'('-')QFactB(J). 

where p is the length of J. 

Note that the formula enumerating type B permutation tableaux is very similar to 
the formula enumerating type A permutation tableaux. 

As an example, suppose we want to enumerate according to rank the type B 
permutation tableaux that have the following shape: 



(113) 



We take n = 4 (we would get the same answer for any n > A), and k = 2 since the 
widest row has width 2. Then the corresponding composition is / = (1,2,0). We 
then get 

PTfi,2,o)(g) = q-'{q-'m2]l{l + g)(l + q') - q-'[2]l{l + q) - [2],(1 + g) + 1) 
= g^ + 4g^ + 8g^ + lOg + 6. 

We will prove Theorem 15.11 directly: we first prove some recurrences for type B 
permutation tableaux, and then prove that the formula in Theorem 15.11 satisfies the 
same recurrences. 
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Lemma 5.2. The following recurrences for type B permutation tableaux hold. 



(114) 
(115) 



PT 



PT 



B 

{«1,«2,«3, 



B 

(0,12,13, ■■■,ik) 



(g) = PT 



B 

{i2,h,--;ik) 



(q), 



i2,h,---,ik) 



(q) + qPT^,^,,. 



2 + l,J3v,« 



kM) 



+ P'^{0,h+i2-l,i3,-,ik)(^) ■ 



Proof - The first recurrence says that enumerating permutation tableaux of a shape 
which has a unique row of maximal width is the same as enumerating permutation 
tableaux of the shape obtained from the first shape by deleting the rightmost column. 
This is clear, since the rightmost column will have only one box which must be filled 
with a 1. 
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PiGURE 4. 











To see that the second recurrence holds, see Figure HI Consider the topmost box 
b of the rightmost column of an arbitrary type B permutation tableau of shape 
corresponding to (zi,...,?^). Since ii > 0, the rightmost column has at least two 
boxes. If b contains a 0, then by definition of type B permutation tableaux, there is a 
1 below it in the same column - which implies that the entire row containing b must 
be filled with O's. We can delete that entire row and what remains will be a type B 
permutation tableau (of smaller shape). 

If b contains a 1 and there is another 1 in the same column, then we can delete the 
box b and what remains will be a type B permutation tableau. 

If b contains a 1 and there is no other 1 in the same column, then let b' denote 
the bottom box of that column. By the definition of type B permutation tableau, 
the entire row of b' is filled with O's. If we delete the entire row of 6' and every box 
below and in the same column as b, then what remains will be a type B permutation 
tableau. h 

Now we prove Theorem 15. 1[ 
Proof - [of the theorem] 
Let 



(116) 



g,(q) := ^(-l/g)^(^)-^(-^)g-«t'(^"^)QFactB(J). 






We want to prove that PT^- (g) = gi{q). We claim that it is enough to prove the 
following two facts: 
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(1) 9{0,i2,i3,-,in)i'l) = 9{i2,i3,-,ir.)i'l) 

(2) 9ih,i2,-,ir.)iQ) = Q9iil-l,i2 + l,i3,-,ini'l) + 9ih-l,i2,i3,-,in)i'l) + 5'(0,il+i2 -l,i3,-,in) (?) 

when ii > 0. 

By Lemma [5.2^ both of these recurrences are true for PTp{q). And the two recur- 
rences together clearly determine gi{q) for any composition /, which is why it suffices 
to prove these recurrences. 

Consider the first recurrence. To prove it, we will pair up the terms that occur in 

(117) 9o,2,...,M ■■= 5^(-l/g)^(^)-^(")g-^*'(^-^)QFact^(J), 

pairing each composition of the form J := (0, ji, J25 • • • , jr) with the composition 

J' ■■= (jl,J2,---,jr)- 

Note that i{J) = i{J') + 1 and st'(/, J) = st'(/, J') + 1. Also 

(118) QFacts(0, ji, J2, • • • , jr) = QFact5(ji, J2, • • • , Jr)(l + ql 
so that 

(119) QFact^(0, ji, j2, ■■■Jr)- QFact5(ji, j2, ■ ■ ■ , jr) = g''QFacts(ji, j2, • • • , jV)- 
And now it follows from the fact that 

(120) st'((0,J),(0,J)) = g'-V(J,J), 

that the contribution to 5'o,i2, ■■■,«« (?') by the pair of compositions J and J' is exactly 
the contribution to gi^,...,i„{q) by the composition (ji,...,>). So fi'(o,i2,J3, .-,««)(?) = 

9(i2,i3,-,in)iq)- 

Now let us turn our attention to the second recurrence. We prove the second recur- 
rence by showing that each term of 9{ii,...,i„){q) comes from either one term each from 
q9{ii-i,i2+i,i3,-,in){q) and 9{ii-i,i2,...,in){(l), OT one term each from qg^i,_i^i,+i^i,^,„^i„){q) 
and g(ii-i,i2,...,i„){q) and two terms from g(^o,h+i2-iA3,-,in)iq)- 

Let us denote the relevant compositions by / := (ii,...,i„), /' := (ii — 1,^2 + 
1, is, ... , in), /" := (^1 -1,12, ■■■, in) and /'" := (0, Zi + Z2 - 2, Zs, . . . , z„). 

First, consider the terms of f7(j^,,,,,j„)(g) corresponding to compositions J such that 
the first part of J is ii, i.e., J has the form (ii, J2, js, •••,>)■ Let us compare this term 
to the terms of qg(h-i,i2+i,i3,...,ir^){q) and g(h-i,i2,...,i„){q) corresponding to the parti- 
tions J' := (zi - 1, j2 + 1, J3, ■ ■ ■ , jr) and J" := {ii - 1, J2, ja, • • ■,jr), respectively. All 
three terms have the same sign and the same st': st'(/, J) = st'(/', J') = st'(/", J"). 
And now it is easy to see that gQFact5(J') + QFact^(J") = QFact^(J): 

q[rY'-^[r - iy^+^[r - 2Y' ■■■+ [rY'~^[r - If^r - 2Y' . . . 

(121) = {q[r - 1] + l){[rY-^[r - lp[r - 2p . . . ) 
= [r]''i[r-lp[r-2p^... 

Note that all terms contain the extra factor YYtZi (1 + ?*)■ Therefore the term corre- 
sponding to J is equal to the sum of the terms corresponding to J' and J". 

Now consider each term of 5'(j^^...^j^)(g) which corresponds to a composition J such 
that the first part of J is not ii, i.e., J has the form (ji, J2, js, • • • ,jr) where ji = 
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H + 'i'2 + ■ ■ ■ + ik where k > 2. Let us compare this to the following four terms: the 
term of 5'fi'(n-i,J2+i,J3, .■■,««(?) corresponding to the composition J' := J; the term of 
g{ii-i,i2,..;in){l) corresponding to the composition J" := (ji — 1, J2, • • • , jr); and the 
two terms of 9(0,11+12-1,13,. ..,i„){'l) corresponding to the compositions J'" := (0,ji — 
1) J25 • • • ,jr) and J^"^^ := J". Note that the terms corresponding to J, J', J", and J^^^ 
have the same sign, while the term corresponding to J"' has the opposite sign. And 
all five terms have the same st' statistic. The quantity QFact^ is nearly the same 
for every term, and if we divide each term by QFact^(J"), it remains to verify the 
equation: [r + l]q = q[r + l]q + 1 — {1 + q^^^) + 1. This is clearly true. 

We have now accounted for all terms involved in the recurrence. This completes 
the proof of the theorem. h 

It is very likely that colored Hopf algebra analogues of WQSym, FQSym, Sym 
already defined in [28l [3ll [29l El [331 132] could be used to justify the g-enumeration of 
permutation tableaux of type B. Based on preliminary calculations, we believe that 
the type B analogue of the matrices from Section 13.8.11 are given by computing the 
transition matrix between the S^ and two new bases \l/f and L^{q). Here 



B . ^/ 



(122) *f . ^r- , 

and L^{q) is defined by having ML(g)^^ as transition matrix from the "^^ . Note also 
that this interpretation would immediately generalize to colored algebras with any 
number of colors and not only to two colors. 

6. Appendix - Conjectures 

We define the descent tops (also called the Genocchi descent set) of a permutation 
(7 G Sn as GDes(cr) := {i G [2,n] | a{j) = i ^ a{j + 1) < o"(j)}. In other words, 
GDes(cr) is the set of values of the descents of a. We also define the Genocchi 
composition of descents GC(cr) as the integer composition I oi n whose descent set 
is{d-l\de GDes(a)}. 

/^From Theorem 5.1 of [H], it is easy to see that Ej{l) is equal to the number of 
packed words w such that 

(123) GC(Std(w)) = / and ev(w) = J. 

So Ej{q) is the generating function of a statistic in q over this set of words. We 
propose the following conjecture: 

Conjecture 6.1. Let I and J he compositions of n and let W"{I,J) he the set of 
packed words w such that 

(124) GC(Std(w)) = I and ev(u;) = J. 
Then 

(125) E/(g)= Yl ^*°''^"^' 

wew"{i,j) 

where totg is the numher of occurrences of the patterns 21 — 1 and 31 — 2 in w. 
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For example, the coefficient 2 + 2g + q^ in row (3, 1) and column (2, 1, 1) comes from the fact 
that the five words 1132, 1231, 1312, 2311, and 3112 respectively have 0, 0, 1, 1, and 2 occurrences 
of the previous patterns. 

There should exist a connection between the sinv statistic and the pattern counting 
on special packed words but we have not been able to find it. 

Note that packed words w are in bijection with pairs {a, J) where cr is a permutation 
and J a composition finer than the recoil composition of a. Since the patterns 31 — 2 
in Std(u7) come from patterns 21 — 1 or 31 — 2 in w, Conjecture 16.11 is equivalent to 

Conjecture 6.2. Let I and J be compositions of n and let P"{I, J) be the set of 

permutations a such that 

(126) GC((t) = / and DC{a-^) ^ J. 
Then 

(127) Ef{q)= Yl ^*°*^'^^' 

weP'{i,j) 

where tot(o") is the number of occurrences of the pattern 31 — 2 in cr. 

If we apply Schiitzenberger's involution to permutations, that is, a ^-^ ujcxuj, where 
uj = n---21 (also known as taking the reverse complement), the statistic descent 
tops is transformed into descent bottoms, and patterns 31 — 2 are transformed into 
patterns 2 — 31. In that case it follows from results of [38] that for J = 1", the sum 
in equation (11271) gives the g-enumeration of permutation tableaux of a given shape. 

Therefore if we assume Conjecture 16. 2^ Theorem 14.21 implies the following. 

Conjecture 6.3. When K = 1", 

(128) Ef{q) = FTfiq) = 5^(-l/g)'(^)-'(^)g-*'(^"')QFact^(J). 

Going from S{q) to R{q) is simple, and allows us to reformulate Conjecture 16.31 as 
follows: 

Conjecture 6.4. Let I and J be two compositions of n. Let PP'(/, J) be the set of 
permutations a such that GC(cr) = / and 'DC{a^^) = J. 
Then 

(129) F/{q)= J2 ^*°*^''^- 

o-ePP(/,J) 

For example, the coefficient 1 + q + q^ in row (3, 1) and column (3, 1) comes from the fact that 
the words 1243, 1423, 4123 respectively have 0, 1 and 2 occurrences of the pattern 31 — 2. 

7. Tables 

Here are the transition matrices from R{q) to L{q) (the matrices of the coefficients 
Ff{q)) for n = 3 and n = 4, where the numbers have been replaced by the corre- 
sponding list of permutations having given recoil composition and LC- composition. 

To save space and for better readability, has been omitted. 
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(130) 



LC\Rec 


3 


21 


12 


111 


3 


123 








21 




6132 
6312 


213 




12 






231 




111 








321 



(131) 



LC\Rec 


4 


31 


22 


211 


13 


121 


112 


nil 


4 


1234 
















31 




61243, 1423 
64123 


61324 
63124 




2134 


2143 






22 






61342 
63142 




2314 








211 






3412 


61432, 4132 
64312 




62413 
64213 


3214 




13 










2341 








121 












62431 
64231 


3241 




112 














3421 




nil 
















4321 
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